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Abstract. We obtain classification, solvability and nonexistence theorems for positive sta¬ 
tionary states of reaction-diffusion and Schrodinger systems involving a balance between re¬ 
pulsive and attractive terms. This class of systems contains PDE arising in biological models 
of Lotka-Volterra type, in physical models of Bose-Einstein condensates and in models of 
chemical reactions. We show, with different proofs, that the results obtained in [ARMA, 213 
(2014), 129-169] for models with homogeneous diffusion are valid for general heterogeneous 
media, and even for controlled inhomogeneous diffusions. 


1 Introduction 


This paper is a sequel to our recent work [5H] , in which we studied stationary states of 
systems of reaction-diffusion PDEs or standing waves of coupled Schrodinger systems, 
including as a particular case the important for applications system 



(a(x)u'^ — c(x)u'^)-b/i(x)] in 0, 
uF {b{x)vF — d{x)v^) -b v{x)\ in fl, 


( 1 ) 


where Ft C 


P>0, q>0, q>\l-p\, (2) 

and the coefficients a, b, c, d, p, v are Holder continuous functions in 0, with 

a, 6 > 0, c, d > 0, ab — cd> 0 in H. (3) 


Cases of particular interest are: 

(i) p = 0, q = 1 - then ([T|) is a Lotka-Volterra system, a model on biological species 
interactions; 

(ii) p = 0, q = 2 - then ([T]) models phenomena in nonlinear optics and the theory 
of Bose-Einstein condensates; 

(iii) p = 1, q = 1 - m this case o is related to a model of chemical reaction. More 
general models in chemistry are obtained by varying p and q. 

We refer to Section 1.3 in [2S] for a more detailed discussion on these applications, 
as well as references. We observe the last condition in ([3]) means the reaction in the 
system dominates the absorption, so there is no conservation of mass in the time- 
dependent version of ©• In |28] we studied the classification and non-existence of 
positive solutions of ([I|) in unbounded domains, as well as the a priori bounds and 
existence results which can be inferred in smooth bounded domains. 

A major difficulty in the study of system ([1]) is that it is in general neither co¬ 
operative (or quasi-monotone), nor variational in the sense that its solutions cannot 
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be written as critical points of some functional defined on a Banach space. The core 
of the new method developed in [28] consists in proving Liouville type theorems for 
a system of elliptic inequalities satisfied by some auxiliary sub- and super-harmonic 
functions of u, v. This method gives many new results even for systems which happen 
to be variational, such as (i) and (ii) above with a = d. 

However, the proofs of the results in [28] depend crucially on the fact that the 
second-order elliptic operator in ([T|) is the Laplacian, that is, in the models above 
only homogeneous diffusion can be considered. Our main goal here is to remove this 
hypothesis and show that the main results in [28| are valid for general operators in 
non-divergence form. It is remarkable that the (necessarily) different proofs we give 
here not only permit to generalize but also to shorten some of the proofs from |28] . 
Our arguments will be entirely based on the maximum principle and its consequences 
from the regularity theory for elliptic equations. 

In terms of the applications, if the underlying stochastic process Xt is not a 
pure diffusion (i.e. a Brownian motion Wt) but rather follows dXt = Ti{Xt)dWt, for 
some positive variance matrix S, in the corresponding PDE the Laplacian is replaced 
by tr{A{x)D‘^u) = ^ aij{x)dijU, where the matrix A = accounts for the spatial 
inhomogeneity. If the process is also allowed to have drift dXt = h{Xt)dt + Tj{Xt)dWt, 
then we end up with the differential operator with a first order term 

Lu = tr{A{x)D‘^u) + b{x).Du = aij{x)dijU -|- bi{x)diU. (4) 

Replacing the Laplacian by such operators in the examples (i)-(iii) above means 
allowing heterogeneous media, as well as a possibility to consider gradient-dependent 
equations, i.e. advection in addition to diffusion and reaction-absorption (for (i) and 
(iii)), or more general derivative Schrodinger equations (for (ii), see for instance [^, 
chapter 1.6). 

Even more generally, an object of intensive study are controlled processes, in which 
Xt follows dXt = batiXt)dt + aat{Xt)dWt, where at is an index process corresponding 
to a choice made in order to maximize or minimize some cost function (see m, m)- 
The PDE operators modeling such processes are suprema or infima of linear operators 
as in with fixed ellipticity constants and bounds for the coefficients. These 
operators are usually referred to as Hamilton-Jacobi-Bellman (HJB) operators, and 
are in turn a subclass of the so-called Isaacs min-max operators, basic in game theory. 
In the following we will consider Isaacs operators as in dSj) below, that is, sup-inf over 
arbitrary index sets of linear operators as in ([3|) 

X[u] := sup inf = sup inf | ^ 4 “’^^ ix)dijU + ^ {x)diU | ; (5) 

here A, B are arbitrary index sets and the coefficients are continuous 

functions. We assume that all eigenvalues of the symmetric matrices belong 

to a hxed interval [A, A], and that the L°°-norms of the vectors are bounded by 

B, for some constants 0 < A < A, H > 0. Note this is equivalent to Definition 13.21 
below. Note also that X is linear and reduces to dH) when |Al| = |H| = I in dSj), 
and that is a HJB operator when |Al| = I or \B\ = 1. In the sequel we will 
write X[u] = F{D‘^u, Du, x) when we want to distinguish the dependence of X in the 
derivatives of u and x. Writing F{D‘^u) will mean F is autonomous and depends only 
on the second derivatives of u. 
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Throughout the paper solutions are understood in the viscosity sense. By applying 
the well-known regularity results for viscosity solutions to the systems we consider, 
we know that their solutions are in for some 7 > 0 , and even in provided the 
operator T is of HJB type and the coefficients in the equation are Holder continuous. 
We observe that viscosity solutions are not an added complication, they provide a 
good framework in this setting, just like Sobolev spaces do for some divergence-form 
operators, even when one knows that any //^-solution is classical. 

Next, we comment on the novelty of our results. First and foremost, they are the 
first of their kind for models like (i)-(iii) above with controlled diffusions, that is, for 
systems like o where the Laplacian is replaced by a fully nonlinear operator such as 
a Isaacs (or even HJB) operator. 

Let us give some more context for each of the models (i), (ii), (hi). Linear operators 
as in Q were considered for Lotka-Volterra systems in a number of papers; the most 
general results available to date as well as references can be found in m- When 
reduced to the linear case, the theorems below strengthen the results from [19] . 

The comparison between the case of the Laplacian and more general operators 
is probably most easily made for (ii). For the last ten years there has been a huge 
amount of work for systems in the form (ii), and they all assume the differential 
operator is in divergence form, in particular the Laplacian. When reduced to (ii), 
our main results from [28] completed the previous works on Schrodinger systems, by 
establishing existence results for the case a, b,c,d > 0 , which was almost unstudied. 
On the other hand. Theorem o below seems to be the first result whatsoever for 
Schrodinger systems governed by inhomogeneous diffusions, i.e. with non-divergence 
form linear elliptic operators, independently of the sign of the coefficients a, b, c, d. 

Finally, for (iii), the results in |28| seem to be the first on their kind. Here we 
extend these to spatially inhomogeneous and controlled diffusions, which have obvious 
relevance in chemistry. 

We observe that existence and non-existence for fully nonlinear systems with a 
different class of nonlinearities (with cooperative and fully coupled leading terms such 
as Lane-Emden type systems, see (I2UD below) can be found in [32]. We refer to that 
paper, as well to [9] and |23|, for more examples and references on problems where 
fully nonlinear systems appear. In passing, and in order to provide a quotable source, 
below we will record several general nonexistence results for systems of Lane-Emden 
type, which essentially follow from a number of recent advances in the theory of Isaacs 
operators but do not seem to have appeared before. We will use some of these results 
in our proofs. 


The following classification theorem is our first main result. It represents a strong 
rigidity property, and states that nonnegative entire solutions (tt, u) of the system 


—F{D‘^u) = u {av^ — cu^) ] 
—F{D‘^v) = V (bu'^ — dv^) ] 


( 6 ) 


can only be semi-trivial (i.e. rt = 0 or u = 0 ) or have proportional components (i.e. 
u/v =const). In other words, rather unexpectedly, the existence of a positive entire 
solution of dGj) is equivalent to the existence of a positive entire solution of a scalar 
equation. 
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Theorem 1.1. Assume a,b,c,d,p,q are real numbers such that and (0) hold. 

If u, V are nonnegative funetions which satisfy in the whole space M”, then 
either u = 0, or v = 0, or there exists a real number K > 0 such that u = Kv. 

A particularly remarkable feature of this result is that it is independent of any 
notion of criticality, that is, of how large p, q oi n are. 

Theorem 11.11 reduces the question of existence of positive entire solutions of the 
system ([ 6 ]) to that of the scalar equations 

- F{D‘^u) = 0 and - F{D^u) = in M”. (7) 

We discuss the available nonexistence results for these equations below. 

Next, we consider the question of existence of positive solutions of ([ 6 ]) in cones 
of M”. By a cone we mean a set in the form = {x G M"" \ {0} : x/|x| G to}, for 
some C^-smooth subdomain of the unit sphere w. 

Theorem 1.2. Assume a,b,c,d,p,q are real numbers such that ^ and ^ hold. Let 
u, V be nonnegative funetions which satisfy & in a nonempty open cone C M"", 
and are proportional or vanish on dC^^. 

(a) If u and v are bounded, then u and v are proportional, or u and v vanish. 

(b) If c = d = 0 then u and v are proportional, or one of them vanishes. 

This result is independent of how large p and q are, and reduces the system to 
the equations dH), set in C^). 

Note that results on classification of bounded solutions of fully nonlinear systems 
in cones were previously obtained in [32], for a different type of nonlinearities (of 
Lane-Emden type), and with different proofs. On the other hand, the statement (b) 
above is to our knowledge the first classification result for unbounded solutions of 
systems with operators in nondivergence form. We also observe this result covers 
nonlinearities such as uv"^, u^v, which were recently found to play an important role 
in some applications, see for instance [Tj. 

Finally, we state the existence results in bounded domains which we can obtain 
as consequences of the previous theorems and well-known techniques from Leray- 
Schauder-Krasnoselskii degree theory. 

We introduce the following notation. For any orthogonal matrix Q and any fixed 
y G n we denote with Fg^y the pure second order operator defined by 

FQ,y{D\) := F{Q^ D\Q,0,y). 

If F is a linear operator as in ([3|), then it is easy to see that for each y there exists 
Q such that FQ^y{D‘^u) = Ari. If F is rotationally invariant (such as, for instance, 
an extremal Pucci operator - see Definition 13.11 belowb then Fg^y = F. Recall that 
F{D‘^u) is rotationally invariant if it only depends on the eigenvalues of D^u. 

In the following we assume that D is a bounded Lipschitz domain such that each 
point y on the boundary dQ has a neighbourhood in Q which is C^-diffeomorphic to 
a neighbourhood of the origin in some closed cone C^^y. Observe if dLl is C^-smooth 
then every such cone is a half-space. 

Theorem 1.3. Let Ll be as stated, and assume the coefficients a,b,c,d,ij,,i',p,q are 
such that (0) and (0) hold and 

inf(a 6 — cd) > 0 , in Q. ( 8 ) 
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Assume in addition that for each y G O there exists an orthogonal matrix Q such 
that if u is a bounded nonnegative solution of the equation 


— FQ^y{D‘^u) = m R” or C^jy, then u = 0, (9) 

(see Provosition \l.l\ below). Then the system fiP, with the Laplacian replaced by any 
general Isaacs operator as in W, has a positive solution in O with u = v = 0 on dfl. 


It is worth observing that we can consider Lipschitz domains almost “free of 
charge”, instead of only smooth domains. This is also a new feature with respect to 
previous works on these types of systems, even for systems with Laplacians. 

The first hypothesis in ([ 8 ]) cannot be weakened, even for simple systems with the 
Laplacian (see the remark after Corollary 1.4 in [5H]). As will be clear from the proof, 
instead of assuming that y, i/ < 0 in ([ 8 ]) we could suppose that /r, ly are smaller than 
the (positive) first semi-eigenvalue of the Pucci maximal operator, or than the first 
semi-eigenvalue of F itself, if T is a HJB operator (Definition 13.,Sp . 

A discussion of hypothesis ([9]) is in order. 

In general, the only non-existence results in unbounded domains for the equation 
in Q concern positive supersolutions. Optimal results for nonexistence of positive 
supersolutions of ([9]) in the whole space or in cones are known, |16] . 12], 131,126], 0, 
and can be expressed in terms of the so-called scaling exponents of the operator 
in Q. These results will be discussed in detail in the next section, we will record 
here their consequences which apply to hypothesis ([9|). 

Given an Isaacs operator F{D‘^u), we denote with a*{F) the ’’scaling exponent” 
of F, defined as the supremum of all positive a such that —F{D^u) > 0 has a (—a)- 
homogeneous solution in R”'\{0} (if such a exist; if not, then even —F{D‘^u) > 0 has 
no positive supersolutions in R""). Similarly, we denote with a^(T, the supremum 
of all positive a such that —F{D‘^u) > 0 has a (—a)-homogeneous solution in C(^\{0} 
(such a always exist) See [5] and 0 for more details on these scaling exponents and the 
way they describe properties of the operators F. Obviously a*{FQ^y) < a^{FQ^y,C). 

It is known that a*(A) = n —2, a'’'(A,R”) = n— 1, and for every Isaacs operator 




n-1 < a+(F,K") < 4„_ 1. (10) 

A 


These bounds are obtained by evaluating the scaling exponents for the Pucci extremal 
operators, and first appeared in m and [25] . 

The following proposition contains necessary conditions for Q, which are direct 
consequences from the Liouville results for the equation in Q in the next section. 

Proposition 1.1. The hypothesis (0) in Theorem 03) is satisfied if 


(i) p + q< 


i^Q,yjCuiy ) 


, y € dTl and p + q < 


max 


{a*{FQ^y),0} 


, y Gil, or 


(a) Q is a -domain, Fy{D‘^u) = F{D'^u,0,y) is a rotationally invariant operator 


for each y gH, and p + q < 


max{Q;*(Fy), 0}' 


or 


(in) H is a 0“^-domain, F is linear as in 0 , and p + q < 


n — 2 
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Observe the upper bounds in this proposition are void, if the scaling exponent 
a* is nonpositive (this is the case for instance for the Pucci maximal operator if 
j > n — 1). For any given operator the scaling exponents can be evaluated in terms 
of (bounds on) the coefficients of the operator, by constructing homogeneous sub- and 
super-solutions, and the bounds thus obtained can be combined with ProDosition ll.il 
The most general bounds are given in (jlOp . and combining (jlOp with Proposition (jl.ll) 
gives explicit relations between A, A, n, p, q, such that Q is satisfied. In the linear 
case, ProDosition ll.il (iii), the result is particularly simple to state. 

The paper is organized as follows. In the next section we state more general 
results which follow from our proofs, and list various non-existence results for elliptic 
inequalities and systems in unbounded domains. Section [3] contains some easy or 
well-known preliminary results; while in Section 0] we prove nonexistence results in 
unbounded domains for a class of systems that will be derived from the systems we 
consider. Section [5] contains the proofs of the classification theorems in unbounded 
domains (Theorems 11.11 and 11.21 and their more general versions, Theorems 12.1112.51 
and ESI) while in Section [ 6 ] we give the proof of the existence result. Theorem 11.31 
(and its more general version, Theorem EH]) • 

2 More general results and Liouville theorems 

Our proofs yield more general results than the ones stated in the introduction. We 
list these more general theorems in this section. We also discuss here Liouville type 
results for scalar inequalities and systems in unbounded domains. 

The system ([^ is included in the class of systems 

f -J^[u] = fix,u,v), xen 
[ —J^[v]=g{x,u,v), X G n, 

where the main feature of the nonlinearities / and g (or their leading order terms) is 
that they satisfy the condition 

3K>0: [f{x,u,v) — Kg{x,u,v)][u — Kv]<0 for all (u,u) G M^, x G 12. (12) 

Indeed, we recall the following result from [28] (Proposition 1.3 in that paper). 

Proposition 2.1 ([28]). If f = u'^vP[av'^ — ctt'?] and g = v'^vP[hu'^ — dv'^], where the 
real parameters o, b, c, d, p, q, r satisfy 

a,b > 0, c,d>0, ab > cd, p,r>0, q > 0, q>\p — r\. (13) 

then f and g satisfy Lfg)) . for a unique number K such that a — cK'^ > 0, bK'^ — d > 0. 

2.1 Liouville theorems in cones 

We assume here that fl is the cone = {tx,t > 0, x G w}, where a; is a C^-smooth 
subdomain of the unit sphere in M”. The following theorem contains Theorem 11.21 

Theorem 2.1. Let J- be an Isaacs operator as in and f and g satisfy m- Let 
Ll = Cuj and {u,v) be a bounded solution of 177]) such that u = Kv on dCuj- 
Then u = Kv in C^j- 
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This theorem will be obtained as a consequence of the Phragmen-Lindelof principle 
for fully nonlinear equations. Theorem l2.1 [ reduces the question of existence of positive 
solutions of m in a cone to the very important in itself question of solving the scalar 
equation 

— F[u] = f{x,u) >0, ti > 0, in Ccj- (14) 

Two types of nonexistence results are available for this equation: general results 
for supersolutions and more precise results for solutions when the domain is a half¬ 
space and the operator J-" is rotationally invariant. As far as the latter case is con¬ 
cerned, it is proved in m and Theorem 3.1 in m that the nonexistence of solutions 
of the equation —F{D‘^u) = f{u) in M"' (and even in implies the nonexistence 

of positive solutions in a half-space of M", for every rotationally invariant operator 
F and locally Lipschitz nonlinearity / (in |31j only Pucci operators were considered, 
but the proof is the same for every rotationally invariant operator). An extension of 
this result to systems is proved in [32] . Even stronger results are known if the elliptic 
operator is the Laplacian, m- 

Next, a nearly optimal result for supersolutions can be obtained by combining the 
results and methods from the recent papers |3] and |1] . In order to provide a quotable 
source, we state a rather general version of this nonexistence theorem. 

Theorem 2.2. Let F{D^u) be an Isaacs operator, and b > 0. Set 

F[u] := F{D‘^u) + ^^\Du\. 

Let a~^ = a^{F,Ci^) > 0 and a~ = a~{F,Ci^) < 0 be respectively the supremum 
and the infimum of all a gM. such that —iF[u] > 0 has a positive {—a)-homogeneous 
supersolution in C^j \ {0} (as in Section 3 of m- Let 7 < 2 and 

2—7 , 2—7 . , 

a":=l +-^< 1 , a+:=l + ^^>l. (15 

a 

Assume that the function g : ( 0 , oo) —>■ ( 0 , oo) is continuous and satisfies 

liminf s~'^^g{s) > 0 and liminf s~‘^ g{s) > 0 , (16) 

s\0 s->-oo 


while the continuous function h : Coj\ (0, oo) is such that for some co' C co and 

Co > 0 


h{x) > co\x\ in C^i \ Br^. 

Then for every Rq > 0 the differential inequality 


- F[u] > h{x)g{u) in \ Br^ (17) 

does not have a positive solution. 

Proof. This theorem is proved by repeating the proof of Theorem 5.1 in [3], replacing 
the functions and T” there by the corresponding functions constructed in |3] and 
by making use of the comparison principle for F. □ 

We list several consequences. 
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Theorem 2.3. Let F{D‘^u) be an Isaacs operator and B >{). Let 

a~ = a~{F,Ci^), and a~^, a~ be defined as in ilT5)] . The differential inequality 

— F{D^u) + B\Du\ > h{x)g{u) in \ Br^ (18) 

does not have a positive solution, provided g and h are as in the previous theorem, 
with m replaced by the slightly stronger hypothesis: for some e > 0 

liminf > 0 and liminf s~'^ ~^9{s) > 0, (19) 

s\0 s—>-oo 

Proof. This is a consequence of the previous theorem, if we observe that for every 
b > 0 there is i?o > 0 such that B < b/\x\ for |x| > Rq and take b so small that the 
scaling exponents of F{D^-) + {b/\x\)\D- \ are sufficiently close to a^{F,Cuj), and (fT 6 l) 
is satisfied. □ 


Next, we give two Liouville theorems for systems in cones. 

The following result, which applies to the systems from the introduction, is an 
obvious consequence of Theorem 12.11 and Theorem 12.31 

Corollary 2.1. Under the hypothesis of Theorem \2.1l if F[u] > F{D‘^u) — B\Du\ 
and there exist cq, u > 0 such that for all x G M” and v > 0, 


f{x,Kv,v) = 


with fj < 1 + 


2 

«+(F,0’ 


then the only nonnegative bounded solution of m in is the trivial one. 


Finally, we record the following Liouville theorem for the so-called fully nonlinear 
Lane-Emden system. More general right-hand sides can be readily studied by the 
same argument (given in Section 6 of [3]). 


Theorem 2.4. Let Fi{D'^u), F 2 {D‘^u) be Isaacs operators with scaling exponents af, 
in a cone C^j. Let r, s > 0. The only nonnegative solution of the system 


-Fi{D^u) + B\Du\ > 
-F 2 {D'^u) + B\Du\ > n" 


in Cuj \ Br^ 


is the trivial one, provided 


( 20 ) 


rs < 1 


or 


2(1+ r) 
rs — 1 


< a. 


or 


2(1 -|- s) 

rs — 1 


< a^. 


If B = 0 weak inequalities can be allowed in the last hypothesis. 

Proof. Repeat the proof on page 2041 in [3] where the whole space instead of a cone 
was considered. Replace the references to Lemma 3.8 there by references to Lemma 
5.4 in that paper, and as above, replace the functions T'*', by the more general 
functions of this type, constructed in [Ij for arbitrary Isaacs operator. If R / 0, 
reason as in the proof of Theorem 12.31 above. □ 


Finally, we state a theorem for nonexistence of unbounded solutions in cones for 
the type of systems we are mostly interested in this paper, of which Theorem 11.21 fbl 
is a very particular case. 





Theorem 2.5. Let p,q,r, s > 0. We assume that f,g satisfy condition for some 
constant K > 0 and that, for some c > 0, 

/(x, u,v) > c vL and g{x, u,v) > c for all u, x > 0 and x G C^)- (21) 

Let {u, v) be a nonnegative classical solution of m in M”, such that u = Kv on dC^j. 
Let F[u] = F{D‘^u) for some Isaacs operator F. 

(i) Either u < Kv or u > Kv in M!(. 


(^^) If 


r < 1 + 




and 


s < 1 + 


a 


+ 


a 


a"' 


a 




or s > 1 + 


a 


a' 


a 


or r > 1 + 




a 


p- 


a 


( 22 ) 


(23) 


then either u = Kv or {u,v) is semi-trivial. Here a'^ = a^{F,Ci^) > 0 and 
oi~ = a~{F,Cuj) < 0. 


2.2 Classification results in the whole space, 

For Ll = M”', we focus on the following system 

-F{D\) = u^vP[avi - cu'i] 

-F{D\) = v^uP[bui - dv’i] 

where we always assume that the real parameters a, b, c, d, p, q, r satisfy the hypothesis 
(fT3]l of Proposition 12.11 (this hypothesis reduces to ([ 2 ]) and ([3]) when r = 1 ). 

Recalling that <y*{F) was defined in the previous section, we have the following 
result, in which Theorem II.11 is contained. 

Theorem 2.6. Let F{D‘^u) he an Isaacs operator, holds, and K be the number 
given by Pronosition lg.il Let {u,v) be a positive viscosity solution of in M”. 

i) Assume that 

2 

a*(F) <0 or 0<r<lH -. 

^ ^ - - - a*{F) 

If p + q < 1, we assume moreover that u and v are bounded. Then u = Kv. 

ii) Assume that 

a*(F)<0 or ( p <—ttw o.nd c,d>0 

If q r < 1, we assume moreover that u and v are bounded. Then u = Kv. 

Observe in this theorem there is no restriction on the total degree a = p+g+r > 0 
of the system (|24D . The exponents p and r separately can guarantee that (|24p has no 
nonstandard solutions. 

An easy consequence is the following Liouville type result for the system (1241) . 
Theorem 2.7. Under the hypotheses of the previous theorem, if ab > cd and 

—F{D‘^u) = has no bounded positive viscosity solution on R”', (25) 

then has no bounded positive viscosity solution in R”. 



on 
on I 


(24) 
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As far as the hypothesis (1251) is concerned, we recall the following result from [3]. 
Theorem 2.8 ([3]). Let F{D‘^u) be an Isaacs operator. The equation 

-F{D^u) = 

has no positive supersolutions in M”" (and even in any exterior domain in M"" j provided 

2 


a*{F) <0 or 0<(T<1 + 


a*{F)' 


It is worth observig that it is an outstanding open question whether the ranges of 
a for which the equation in ([9]) (for instance, if T is a Pucci operator) does not admit 
entire positive supersolutions or entire positive solutions are different. Note this fact 
is well-known for the Laplacian - the equation —Art = does not have positive 
entire solutions if and only if a < (n -|- 2 )/(n — 2 ), while this equation does not have 
positive entire supersolutions if and only if ct < n/(n — 2). For Pucci operators the 
only result in that direction is [ 2 U| . and it concerns only radial (super)solutions. 

We also record the extension of the last theorem to systems of Lane-Emden type, 
proved in Section 6 of [3]. 

Theorem 2.9. Let Fi{D‘^u), F 2 {D‘^u) he Isaacs operators with scaling exponents a\, 
0.2 in M”. Let r,s > 0. The only nonnegative solution of the system 

I -Fi(D^u) > o’" 

\ -F2{D^u) > u" 

is the trivial one, provided a\ < 0 , or < 0 , or 


in 


(26) 


rs < 1 


or 


2 ( 1 + r) 


< a; 


or 


2(1 -|- s) 


< 02 - 


rs — 1 rs — 1 

In the last theorem in this section we discuss the classification of nontrivial non¬ 
negative solutions of (1231) . 

Theorem 2.10. Under the hypothesis of Theorem \2.1\ if we moreover assume that 
q + r > 1, then any nonnegative bounded viscosity solution of [Ef\) is semitrivial, i.e. 


{u,v) = {Ci,Q) or (u,u) = ( 0 ,C 2 ) 


with Cl, (72 > 0. Moreover: 

- If r = 0, then (u, v) = (0,0). 

- If r > 0, p = 0 and c > 0 (resp. d> 0), then {u,v) = (0,0). 


2.3 A priori estimates and existence in a bounded domain 

We consider the following system with general lower order terms 

—F[u] = u^v^[a{x)v'^ — c{x)u‘’~\ + hi{x,u,v), x £ Ll, 

< —F[v]=v^u^[b{x)u'^ — d{x)v^]+h 2 {x,u,v), x £ Ll, (27) 

u = V = 0, X £ dLl, 

where W is a general Isaacs operator as in ([5]) and 11 is a bounded Lipschitz domain 
such that each point y on the boundary dLl has a neighbourhood in H which is 
C^-diffeomorphic to a neighbourhood of the origin in some closed cone Ct^y. 

The following theorem contains Theorem 11.31 as a very particular case. 
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Theorem 2.11. Let J- be an Isaacs operator, and p,r > 0, q > 0, q > \p — r\, 
q + r > 1. Assume that the system (EP has no bounded positive viscosity solution in 
M” or in any cone as in the definition of the Lipschitz property of H above (sufficient 
conditions for this are given in the previous subsection). 

1. Let a, b,c,d£ C{Ll) satisfy a,b > 0, c,d>0 in LI and 


inf [a{x)b{x) — c{x)d{x)] > 0 . 


(28) 


Let hi,h 2 & C{Ll X [0,oo)^) satisfy 

hi{x,u,v) • 1 o 

lim —--—= 0 , ^ = l, 2 , 

u+v^oo [U + vffi 

and let one of the following two sets of assumptions be satisfied: 


(29) 


r < 1 , and, setting m := minjinf a{x), inf b{x)} > 0 , 

xGfl xGfl 

. hi{x,u,v) _ 

limini -—— > —m, 

i;^oo, u/v^O 11^ 


h2ix,u,v) 
u—)-oo, v^/u^O yp+q 


lim inf 


> —m, 


(30) 


or 


m := mini inf c(x), inf d(x)} > 0 , and 
xgq xm 


hi{x,u,v) 

iimsup -;- < m, 

-^oo, v/uUo 


h 2 {x,u,v) 

limsup -;-< m 

-^oo, u/v-,0 V^+^lvP 


(31) 


(with uniform limits with respect to x € LI in IgPD - 

Then there exists M > 0 such that any positive classical solution {u, v) of (5^ satisfies 


sup u < M, sup V < M. (32) 

p n 

2. Assume that hold, a,b,c,d,hi,h 2 are Holder continuous and for 

some e > 0 


inf u ^ hi{x,u,v) >—oo, 

xG.fl, u,v>0 


sup u ^ hi{x,u,0) < \f ,Ll), 

xGfi, w>0 


inf V ^ h 2 {x,u,v) >—oo, (33) 

xGfl, u,v>0 

sup v~^ h2{x,0,v) < Xf ,Ll), 

xGfl, v>0 


(34) 

sup (max{u, u}) ^ hi{x,u,v) < Xf ,Ll), i = l,2. (35) 

where is the Pucci maximal operator (see Defimtion \3.1\ below). 

Then there exists a bounded positive classical solution of /[5^) . 


Remark 2.1. If H is a HJB operator we can replace the first eigenvalue of the Pucci 
operator by the first eigenvalue of T in the above theorem. 


3 Preliminaries 

For the convenience of the reader, we begin by recalling some definitions. 
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Definition 3.1. Let 0 < A < A. The extremal Pucci operators are defined by 


/ii>0 Mi<0 


sup tr(AM), 
\I<A<AI 


M-{M) = -M+{-M), 


for any symmetric matrix M E Sn, where {ni)i=i..n are the eigenvalues of M. 

Definition 3.2. iF is an Isaacs operator if the following conditions are satisfied : 

• F is uniformly elliptic and Lipschitz: there exist A > X > 0, B > 0, such that 
for all symmetric matrices M, N, and all p,q G x G Q, 

(Hi) M-{M-N)-B\p-q\ < F{M,p,x)-F{N,q,x) < M+{M-N)+B\p-q\, 


• F is 1-homogeneous: for all t >0 and M G Sn, p G x G LI, we have 
{H 2 ) F{tM,tp,x) = tF{M,p,x). 


We will also need the definition of the principal half-eigenvalue of an Isaacs oper¬ 
ator. See [I] for more details. 

Definition 3.3. Let LI be a bounded domain of ML and F be an Isaacs operator. We 
define the finite real number 

xt{-F,n) = sup{/i E M, 3u G C{Ll), u > 0, —F[u] > p,u in fl}. 

We recall all equalities and inequalities in this paper are understood in the vis¬ 
cosity sense. For the notion of viscosity solution, we refer the reader to [2], |10j . 

We recall a transitivity result, whose proof is a simple consequence of (Hi) above 
and Lemma 3.2 in [T]. 

Lemma 3.1. Let F be an Isaacs operator. Assume that f,g G C{L}), u,v G C{L}) 
are viscosity solutions in LI of 

I > /, 

1 m < 9- 

Then w = u — v is a viscosity solution in of 

M+{D^w) + B\Dw\ >f-g. 

The next simple lemma is helpful in exploiting condition (|12l] on the nonlinearities 
/ and g of the system, as shown in the subsequent result. 

Lemma 3.2. Assume that F satisfies {Hi) and ^"[0] = F(0,0, x) = 0. If w,h are 
continuous functions such that sign{w)=sign{h) and w is a viscosity solution in O of 

I F[w] > h 
\ F[-w] > -h, 

then |t(;| is a viscosity solution in Lt of 

J-[|u;|] > \h\ 
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Proof. Assume (f G (7^(0) touches by above |t(;| at xq G If w{xo) > 0, then 
h{xo) > 0 and since f touches w by above at xq, we have 

F{D‘^4>{xo),D(t>{xo),xo) > h{xo) = \h{xo)\. 

If w(xo) < 0, then h(xo) < 0 and since 4> touches —w by above at xq, we have 

F{D'^f{xo),D(t>{xo),xo) > -h{xo) = \h{xo)\. 

If w{xo) = 0, then /i(xo) = 0. Moreover, 4>{xo) = 0 and (/) > |rc| > 0 so xq G 
is a minimum point of </>. Hence D'^cP^xq) > 0, D(j){xo) = 0, from which we deduce 
F{D'^(t){xo),D(j){xo),xo) > F(0,0,xo) = 0 = |h(xo)|. □ 

Lemma 3.3. Assume that F is an Isaacs operator. Let (u, v) be a viscosity solution 
of 07]) in Q and assume that the nonlinearities f,g ^ C{Ll) satisfy /f71|) . Then 

M~^{D‘^\u - Kv\) + B\D\u -Kv\\>\f - Kg\ in H 

in the viscosity sense. 

Proof. Apply Lemma 13.21 to 


w = u — Kv, and h = Kg{-,u,v) — f{-,u,v). 

Observe that condition ( 1121 ) means that hw > 0. Also, by the continuity of f,g 
and m, it is easy to see that for all x G H and x G M, we have Kg{x, Kv, v) — 
/(x, Kv, v) = 0, hence if u; = 0 then h = 0. □ 

The last lemma will be useful when considering system (I24li on the whole space 
(since the auxiliary function Z = inm(u, Kv) will play a crucial role in our analysis). 

Lemma 3.4. Assume that F is an Isaacs operator. Let LI he an open set and let 
u,v,f,g,h G C{Ll). Assume that u and v are respectively viscosity solutions of 


> / 


and 


- > 9 


and that 

j f > h on {u < v} 

] g > h on {u> v}. 

Then w := min(n, x) G (7(0) is a viscosity solution in Ll of 


(36) 


-F[w] > h. 

Proof. Assume (j) G (7^(0) touches by below tx at xq G O. Then f < u and (f <v. If 
u{xo) < x(xo) then w(xo) = u(xo) and (j) touches u by below at xq, hence 

-F{D^(f>{xo),D(j){xo),xo) > /(xo) > h{xo). 

Similarly, if x(xo) > x(xo) then tx(xo) = x(xo) and f touches x by below at xq, hence 

-F{D‘^(I){xo),D4){xq),xq) > g{xo) > h{xo). 


□ 
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For the reader’s convenience, we next recall some known facts to which we refer 
in the subsequent proofs. 

Lemma 3.5. Let F{D^u) be an Isaacs operator. If u is a viscosity solution of 
F{D‘^u) = 0 on M” and u is bounded from below, then u is constant. 

Proof. The proof is well known and is the same as for the Laplacian. The result is 
also included in Theorem 1.7 in [5]. □ 

Lemma 3.6. Let F{D‘^u) be an Isaacs operator. Let z a viscosity solution on M”" of 

-F{D‘^z) > 0. 

i) Assume a*{F) <0. If z is bounded by below, then z is constant, 
a) Assume a*{F) > 0. Then for some m > 0 

TTl 

z > , , for all \x\ > 1 . 

- |a;|o*(F) ■' ' ' - 

Proof. The statement in i) follows from Theorem 4.3 in [3], whereas ii) is a particular 
case of Lemma 3.8 in m- □ 

Lemma 3.7. Let Q be an domain o/M"' and let F{D‘^u) he an Isaacs operator. Let 
/i > 0 and u > 0 be a viscosity solution in Q of 

-F{D^u) > h. 

Then, for any compact K C Ll, there exist 7 = 7 (A, A, n) > 0, c = c(A, A, n, K.Ll) > 0 
such that ^ 

inf n > c . 

Proof. This is proved in [25]. The result in that paper is stated for a linear operator, 
but the same proof applies to any uniformly elliptic Isaacs operator. □ 

It is well known (see for instance [Hj, [101 Proposition 2.9], [111 Theorem 3.8]) 
that it is easy to pass to uniform limits with viscosity solutions. 

Lemma 3.8. Let Q be a domain of and F be an Isaacs operator. Let (uj) a 
sequence of viscosity solutions of 

—F[uj\ = fj{x) in Pi 

where fj G (7(12). Assume that Uj —)• u and fj —)• / locally uniformly in Pi. Then u is 
a viscosity solution of 

—F[u] = f{x) in Pi. 

We also recall the following solvability result. 

Lemma 3.9. Let F be an Isaacs operator, R > 0 and f G Then there exists 

a viscosity solution u G C{Br) of the problem 

f = f{x) X G Bn, 

I n = 0 X G dBn 
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(37) 



as well as a unique viscosity solution u G of the problem 

-M^{D‘^u) +B\Du\= f{x) xeBn, 

u = 0 X G dBn 

Proof. This result is contained in m, US]. 

4 Liouville theorems for weighted inequalities on 

In this subsection, we present two Liouville type results for inequalities on M"'. The 
first one concerns a coercive inequality and extends Lemma 3.4 from [28]. That lemma 
for the Laplacian was proved in |28| by using spherical means - a tool obviously not 
available for more general operators. Here we give a different and more general proof, 
based on estimates from the regularity theory of elliptic equations, such as growth 
lemmas and Harnack type inequalities. 

Lemma 4.1. Let F{D^u) be an Isaacs operator and w > 0 be a viscosity solution of 

F(D^w) > —wP on M”, (39) 

1 + |x|^ 

where p > 0 and H > 0 . 

i) If w ^ 0, then w is unbounded. 

ii) If p > 1, then w = 0. 

Proof. By (Hi) the function w is a viscosity solution on M” of 

M^{D‘^w) > — ^ „ wP. 

1 + \x\^ 

Proof of i). Assume w 0. We define M (R) = sup w, and will show the existence 

Br 

of c > 0 and Rq > 0 such that for all R > Rq 

M{2R) > M{R)+c, 

which implies the statement of i). 

For all X G M”, we define 

= 1 ^\x\2 

Since / > 0 and / G thanks to Lemma [T9l there exists a unique viscosity 

solution Ur > 0 of 

= / on B 2 r 

UR = 0 on dB 2 R. 

We define on B 2 R the function vr = M{2R) — ur, which satisfies 

M~^{D‘^vr) = / on B 2 R 

vr = M{2R) > w on dB 2 R. 
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( 38 ) 

□ 






By the comparison principle we obtain w < vr on B 2 R, which implies 


miuR + M{R) < M{2R). 
Br 


Now, we define on B2 the function ur{x) = ur{Rx), which is a viscosity solution of 

A R^ 

— M~ {D'^ur) = - „ „ w{RxY > e w{RxY on B 2 , (40) 

1 + R‘^\x\^ 

for some e > 0. Since rc / 0, there exists Rq > 0 such that sup w{Ro-) > 0. 

Since > 0, by the local maximum principle applied to w{R-) > 0 for 

R > 0 (see [TUI Theorem 4.8 (2)]), for any q > 0 there exists C = C{q) > 0 such that 
for all R> Ro, 


\\w{R-)\\Rq/R^\ > C sup w{R-) > C sup w{Rq-) = C > 0. 

Si Bi 

However ur> 0 satisfies (SQD, so by the quantitative strong maximum principle (see 
Lemma 13.71) applied to (|T0]l , there exist g'o > 0 and cq > 0 such that 

inf UR>coe \\w{R-Y\\lio{Bi)- 

We choose q = qop and obtain a constant c > 0 such that for all R> Rq we have 


inf -Ur > c. 

Si 

Since inf ur = inf ur, this implies 

Si Br 

M{2R) > M{R)+c. 

Proof of a). We will show that w is bounded on M”, which implies tc = 0, by i). 
This can be proved similarly to [28], we include the full proof for completeness. 

As in [29], we dehne the function wr G C'^{Br) by 

for all X G Br, where a = It is easy to see, by direct computation, that if C > 0 
is large enough then for all x G Br 

A Awr < P wrP (41) 

1 + |x|^ 

in the classical sense. See for instance the computation on page 15 in [28] . 

Note Wr is radial and its first and second radial derivatives are nonnegative. Then 
Wr is convex, so M.^{D'^wr) = A Awr. Hence 

M+{D^wr) < —^wrP. (42) 

1 + |x|^ 

Since wr{x) —> + 00 , there exists R' < R such that wr > \\w\\ ^00 on BrXBr^. 

x^OBr ' ' 

Assume that sup [rc — rcij] > 0. Since w < wr on OBri, this supremum is attained at 
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some xq G Bri. On the other hand wr € C^{Bri), so wr is a legitimate test function 
for (f3^ . and by the definition of a viscosity subsolution we get 


A A 

M"^{D‘^wr{xq)) > ———^ wix^y > ———^ wr{xoY, 

l + Fol l + Fol 

a contradiction with (I42|) . This implies that w < wr on Br/ and then on Br. 

Now, for any x G M”, we let R —>• +oo, and obtain w{x) < limK_).oo = C. 

Hence w is bounded on M"'. □ 


Lemma 4.2. Assume that F{D‘^u) is an Isaacs operator. Let V G V ^ 0 be 

such that for all j > 0 

liminf > 0. 

fl^-+oo BF Jb2r\Br 

Assume that 

a*(F)<0 or 0<r<l + ——. (43) 

a*[F) 


Let z > 0 be a viscosity solution of 


-F{D^z)>V{x)zy xGM”. 


Then z = 0. 

Proof. The case cx*{F) < 0 is obvious since the only viscosity solutions of —F{D‘^z) > 
0 are the constants (see Lemma ITHI ill and because H > 0, V Y 0. Hence we can 
suppose that a*{F) > 0. 

Assume for contradiction that z Y 0. Since z > 0 and —F{D‘^z) > 0, by the 
strong maximum principle (see m Proposition 4.9]) we have z > 0 on M”. 

First, we note that the hypothesis on V implies, for each 7 > 0, the existence of 
Rq = i?o( 7 ) > 0 and cq = 09 ( 7 ) > 0 such that for all R > Ro, 

[ f V{Rxydx\ > Cq. 

\Jb2\Bi J 

Let 7 = 'y{F, n) > 0 be the constant given in the quantitative strong maximum 
principle (see Lemma ISTT]) and Rq = Roi'l)- Let R > Rq. We set zr{x) = z{Rx) and 

miR) = inf z = inf zr. 

Br Bi 

It is easy to see that zr is a viscosity solution of 

-F{D‘^zr) > R^V{Rx){zrY, x G 

By the quantitative strong maximum principle (see Lemma E31), there exists C > 0 
such that 

inf ZR>C \\R^V{R-)zr^\\li(Bi) > CR^m{RY ( [ V{RxYdx 

\Jbi 

Hence, for all R > Rq 

m{R) > CR^co{'y) m{RY■ 
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From this point on the proof is similar to the one given in [3] and in [28] for the 
case when the elliptic operator is the Laplacian. Thus we only give a sketch of the 
proof, with more details in the third case below, where some differences appear. 
First case. Assume 0 < r < 1. For all R > i?o, since m{R) > 0, we have 

> m{R)^~'^ > CR^coi'j) 

because R i—)• m{R) is nonincreasing. We then obtain a contradiction when R —>■ oo. 

Second case. Assume 1 < r < , which is equivalent to > a*{F). 

From the argument used in the previous case, we deduce that for all R > Rq 

m{R) < CR~^^, 


for some C > 0. On the other hand, for any R> 1, 

m{R) > cR~°‘*^^\ 


for some c > 0 (this follows from Lemma 
contradiction, by letting R +oo. 

Third case. Assume r = which is equivalent to 

consequence of this equality, if we set 


ii)). The last two inequalities yield a 
^ = a*{F). As a 


zr = zr, and m(i?) := inf ^ = inf 

and we can check that 

^ < m{R) < C. (44) 

It is easy to see that zr is a viscosity solution of 

-F{D^zr) > V{Rx)z^r, X G M ". 

Hence —F{D‘^zr) > 0 so, by Lemma iTBl iii. we deduce that 

Zr — m(i?)4> > 0 on]R"'\i?i. (45) 

Since F{D‘^{rh{R)^)) = rh{R) F{D^^) = 0, and 

F{D^zr) < -ViRxyz^R, 


by applying Lemma [3T] we get 

-M- {D^ [zR - m{R)^]) = M+ {D^ [m{R)^ - zr]) > V{Rx)z'^r. 

We apply the quantitative strong maximum principle (Lemma 13.7p to the operator 
Ai~ with H = H 5 \ Hi and K = H 4 \ H 2 . We find 7 “ > 0 and c_ > 0 such that 

inf [zr — m{R)^] > c- i [ V{Rxy zr{xY^ dx 
Ba\B2 \JBa\B2 



By the [—a*(F)]-homogeneity of we have > ci on H 4 \H 2 , which implies, by (I44p 
and (1451) . that 

Zr > C 2 on H 4 \ B 2 . 
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Hence we obtain 


inf [zr — fh{R)^] > C 3 i /_ V{Rx)'^ dx 

B4\B2 \JR^\B2 j 


> C4 > 0 


for R large enough, from the hypothesis on V. Therefore, for R large enough and for 
all \x\ = 1 we have 

zr{2x) > rh(i?)$( 2 x) + Cl, 


from which we deduce 


rh{2R) > m{R) + 


Ml ’ 


so m{R) is unbounded, a contradiction. 


□ 


The previous theorem can obviously be applied to any constant function V, but 
also for any nontrivial nonnegative subsolution, as we show in the following lemma. 

Lemma 4.3. Let F{D^u) he an Isaacs operator. Let V € C{W^), V ^ 0 be such that 


F{D‘^V) > 0 


on 


Then, for any 7 > 0, 


lim inf f 

R-^+00 R^ J £ 


> 0 . 


B2r\Br 


Proof. This is a consequence of [IDl Theorem 4.8 (2)]. Indeed, if we apply the latter 
to Vr = V{R-) > 0 for i? > 0, then for any 7 > 0 there exists C = 0 ( 7 ) > 0 such 
that for all i? > 0 , 


R'^ 


1/7 


B2r\Br 


f V{Rxydx\ >C sup H > C sup H = CsupH, 


where the last equality follows from the comparison principle and F{D‘^V) > 0. But 
since V ^ 0, there exists Rq > 0 such that sup H > 0. □ 


5 Proportionality results for systems on or a cone 

5.1 Case of a cone 

In this subsection, we fix a cone = {tx,t > 0, x G tc}, where w is a C^-smooth 
subdomain of the unit sphere Si. We will use the following notation 

B+ = BRnc^, s+ = Si?nC, 

and, given an Isaacs operator F{D‘^u), we will denote by G C \ {0}) the 
positive solutions of 


= 0 , in 

= 0 , on dCi^ \ { 0 } 


(46) 
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normalized so that 'I'^(xo) = 1 for some given point xq G C^, and such that 

'h^(x) > 0 on Cuj and = t°‘ for all t > 0, x G Cuj. 

Here a~^ = a'^{F,Cuj) > 0 > a~ = a~{F,Cuj) are uniquely determined. For more 
details on these functions and their construction, see [1]. Recall that when F is 
the Laplacian and is the half-space {xn > 0}, we have and 

= Xn- 

We next recall the following Phragmen-Lindelhof principle for fully nonlinear 
equations, which is a particular case of the results in [12] or [3j. 

Lemma 5.1. Assume that F is an Isaacs operator. Let w G 
viscosity solution of 

F[w\ >0 on Ccj 
w <0 on dCu). 

Then w < 0 in Cuj. 

Proof. This is a special case of m Theorems A] or |H Theorems 1.6-1.7] and their 
proofs. Using the notation of [1], we set U = U' = so V = C^j and we choose 
V = Cuj. Since w is bounded and «“ < 0 < ., then condition (1.12) of [H Theorem 

1.7] is clearly satisfied. □ 

Proof of Theorem \2.1[ By combining LemmaElSjand Lemma fS.ll we get \u—Kv\ < 0 
in Cco, i.e. u = Kv. □ 

We now turn to the proof of Theorem l2.5l The proof of the corresponding result for 
the Laplacian in |28| depended heavily on the use of half-spherical means. Obviously, 
this tool cannot be used for more general operators, so a different proof is needed. 
Our proof here is shorter and uses the maximum principle and the boundary Harnack 
inequality. 

Proof of Theorem A2.5\ (i). We define the quotient 

qu{r) =inf-^. 
s+ 'k 

Observe that the Hopf lemma applied to —F[u\ > 0 implies > c,. > 0 on Sif OSCoj) 
while the boundary Lipschitz estimates applied to —= 0 imply < Cr on 
Sif n (here u denotes the interior normal to therefore qu{r) > 0. 

Since = 0 on dC^j and tt > 0 we have u > q{r)'I/~ on the boundary dBf'. By the 
comparison principle we have this inequality in Bif. In other words, quir) = iiif^+ 
Therefore the function q^ is decreasing in r, and hence has a limit as r ^ oo, which 
we denote with > 0. In particular, we have u > Ln'h” in 

Similarly, we dehne L,; > 0 such that v > in Now, if both > 0 and 

L„ > 0 we get from (fTT]l 

-F[u] > c(T-)'" 

for some o" > 0. This contradicts Theorem 12.21 since T” > c|x|““ in any proper 
sub cone of C^). 

Let us assume Lu = 0. We are going to prove that u < Kv. Set w = {u — Kv)+. 
Then we know that 

—F[w\ < 0 < —T[u] and w <u in C^)- 


C{Ci^) he a bounded 

(47) 

(48) 
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Let zr be a solution of 


F[zr] = 0 in 

zr = w on 


aB+. 


By the global C^’“-estimates (see for instance Proposition 2.2 in 0) we see that the 
sequence zr is locally uniformly bounded as ii —oo in each hxed compact of C^j- 
Therefore we can pass to the limit and get a function 2 such that 


w < z <u and F{D‘^z) = 0 


in Cj. 


We now define 


w 


Qw{r) := sup—, 


s+ 


T- 


and we see that Q«,(r) is increasing in r, by an argument similar to the one we used 
for qu. 

On the other hand 

in z z ij 

Qniir) = sup -^ < sup — < Cinf — < Cinf — = Cquir), 

where we used the boundary Harnack inequality for the functions ^ et - see for 
instance Proposition 2.1 in [3]. Now limr_>.oo = 0 means the nonnegative 

increasing function tends to zero as r — 00 , that is, Qw = 0 . 

Proof of Theorem \2.5\ (ii). To deduce (ii) from (i) we use exactly the same argu¬ 
ment as in the proof of Theorem 2.8 in [25j, replacing the reference to Lemma 3.1 
there by a reference to Theorem 12.21 above. Observe the conditions (2.6) and (2.7) in 
Theorem 2.8 in [55] are exactly our (l55]l - (f55D with Q;+ = n — 1, a“ = —1. □ 


(49) 


5.2 Case of the whole space 

In this section, we focus on the system 

j —F{D‘^u) = v'P[av^ — cu^] 

[ -F{D\) = v^uP[bu’^ - dv’i] 

In our study of ()3^ we always assume that the real parameters a, b, c, d,p, q, r satisfy 

a,b>0, c,d>0, p,r>0, q > 0, q>\p — r\. (50) 

The last hypothesis provides the following result, proved in the appendix of 


on 


on 


Proposition 5.1. Assume f50l) . 

(i) Then the nonlinearities f and g in the system ()39]l satisfy (fT 2 ]) for some K > 0. 

(ii) Assume moreover that ab > cd. Then the number K > 0 is unique. We have 
K = l if and only ifa + d = 6 -|-c and K > 1 if and only if a + d > h + c. In addition, 
if ab > cd (resp. ab = cd), then a — cK‘> > 0 (resp. = 0 ), bK^ — d > 0 (resp. = 0 ). 


As in |28] . in what follows we set 

Z = mm{u, Kv), and W = \u — Kv\ 
and we establish a system of elliptic inequalities satished by Z and W. 
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Lemma 5.2. Let F{D‘^u) be an Isaacs operator. Assume that IlL^) holds and let 
{u, v) be a positive viscosity solution of Assume that ab > cd. 

a) The functions Z and W are viscosity solutions on M"" of 

-F{D^Z) > 0, M^{D‘^W) > 0. 

b) If p + q < 1, suppose in addition that {u,v) is bounded. Then Z is a viscosity 
solution of 

- F{D‘^Z) > CW^Z^ in M", (51) 

where C > 0 and /3 := max(p + g, 1 ). 

c) Assume r > p and c,d>0. If q + r < 1, suppose in addition that {u,v) is 
bounded. Then W is a viscosity solution of 

M + {D^W)>CZPW^ m M", (52) 

where C > 0 and 7 := max(g + r, 1). 

d) Assume ab > cd. Then Z is a viscosity solution of 

-F{D‘^Z) > CZP+'i+^ in M”. 

Most of the arguments in the proof of this lemma are similar to [28] (except in 
c)), thanks to Lemma 13.31 and Lemma 13.41 We sketch the proof here. 

Proof, a) By Proposition 15.11 we have 

a>cK‘i, bK‘1 > d. (53) 

Hence, on the set {u < Kv}, we have 

f{u,v) = u^vP[avP — cn'^] > cu^yPKKvy — > 0 

and similarly on the set {u > Kv}, we have g{u,v) > 0. Now, we apply Lemma 13.31 
and Lemma Ea to u and Kv with h = 0 and deduce a). 

b) We have — y^ > CqxP~^{x — y), for any real x > y > 0, where Cg = 1 if 
g > 1 and Cg = g if 0 < g < 1. Using (f55|l . on the set {u < Kv} we have 

f{u, v) = u''vP[av^ - cu’i] > ^ u^vP[{Kvy - u^] 

> ^u''vP+^-^{Kv -u)> ■^^^uyKvf+‘^-^{Kv -u)> C^Z^W^ 

for some Ci > 0 (we use that if p + g > 1, we have {KvY~^^~^ > {Kv — uY~^p~^ ., 
whereas if p + g < 1 , then {KvY^^~^ > Cq for some Cq > 0, since v is assumed 
bounded). Similarly, on the set {u > Kv}, we have g{u,v) > C 2 Z^W^, for some 
C 2 > 0 . 

Now, we apply Lemma r3.4l to u and Kv with h = CZ^W^ where C = min(C'i, iLC' 2 ) 
(since —F{D'^[Kv]) = Kg{u,v)) and obtain that Z = min(u, iLn) is a viscosity solu¬ 
tion of 

-F{D^Z) > CZ^Wf^. 
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d) Since ab > cd, we know from Proposition 15.11 that for some small e > 0 we 
have a > cK^ + e, bK'i > d + e. Hence, on the set {u < Kv} we have 

f{u,v) = - cu’i] > + cvrv'P[{KvY - u^] 


and, similarly, on the set {u > Kv} we have the same inequality for g. 

We again apply Lemma 18.41 to u and Kv with h = where we set C = 

emm{K~P~'^, and obtain that Z = min(u,iLu) is a viscosity solution of 

-F{D^Z) > 

c) Thanks to Lemma 7.1 i) in [28] . we know that, since r > p and c,d>0, we 
have for some Cq > 0 

\Kg{u, v) — f{u, u)| > Co uFv^{u + Kvy^'^~^~^\u — Kv\. 

We also note that for x,y > 0 and x + y > 0, we have 

xy 1 

—^— > -mm(x,y). 
x + y 2 


Hence, 


\Kg{u,v) - f{u,v)\ > ^ 


u Kv 
u + Kv 


{u + KvY~^'' ^\u — Kv\ 


> 


Co 


{2K)P 


ZP{u + Kvy^^-^\u- Kv\ >CZPW^, 


for some C > 0 (using that if g + r > 1 , we have {u + Kv)‘^~^^~^ > \u — Kv\'^~^^~^, 
whereas if g + r < 1 , we have (u + Kv)'^~^^~^ > Cq for some Cq > 0 , since u and v are 
assumed bounded). 

Now, thanks to Lemma 1,1.31 we obtain that in the viscosity sense 


M^{D^W) > CZPW'^ in R”. 


□ 


We now can give the following 

Proof of Theorem \2.b\ Let {u,v) be a positive viscosity solution of in R"". Let 
Z and W be defined as in the previous lemma. 

Proof of i) Assume that W 7 ^ 0. From Lemma 15.21 bl. we know that Z is a 
viscosity solution of 

- F{D‘^Z)>CV Z'" in R”, (54) 

where C > 0 and V = W^, with (5 := max(p + ( 7 ,1). From Lemma 15.21 al. we know 
that IF is a viscosity solution on R"" of {D'^W) > 0 so, since /3 > 1 , F is a 
viscosity solution on R” of 

M + {D^V) > 0 . 

Moreover, F ^ 0, hence, by Lemma 14.,11 F satisfies the hypothesis of Lemma 14.21 
Therefore Z = 0, a contradiction since tt, u > 0. Then IF = 0, i.e. u = Kv. 
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Proof of ii) We can assume 

g + r > 1 . 

Q* (I 2 

Indeed, if g + r < 1, then u, v are bounded and r<g + r<l< ^ so the result 
follows from a), which we already proved. 

We can also assume that 

r > p. 


Indeed^ if p + g < 1, then p + g<l<g + rsor>p;ifp + g>l, we can assume 
r > (or else the result follows from a)), so r > > p. 

Since Z is a viscosity solution on M"' of —F{D‘^Z) > 0, by Lemma 13.61 iil there 
exists m > 0 such that 


ZP > 


m 


p 


x\pa*iF} 


for all X G M"" \ Bi. 


Therefore since p < there exists > 0 such that 

,o for all X G \ Bi. 

1 + |xp 

Moreover, > 0 is continuous on B 2 so there exists ^2 > 0 such that Z^ > 
for all X G Bi. 

Therefore, since r > p and c,d>0, by Lemma 15.21 cl VL is a viscosity solution of 

M^(D^W) > —in R*", 

1 + |xp 

for some j 4 > 0 and 7 = g + r > 1. Then, by Lemma l4.ll we have IT = 0. □ 

Proof of Theorem Let (rt, v) be a bounded positive viscosity solution on R” of 
(149 D . Then from Theorem 12.61 we know that u = Kv. Hence u is a solution of 

-F{D^v) = KP{hK<i - d)v^. 

But since ab > cd, we know from Proposition 15.11 that bK'^ — d > 0, so, by using the 
scaling of the equation and the hypothesis, we get u = 0 and hence u = 0. □ 


Proof of Theorem \2.1(A . Let (n, v) be a nonnegative bounded viscosity solution of 
(|i9|) . First note that u is a viscosity solution of 

-F{D‘^u) + Cu>Q 

where C = cv^ > 0. We can apply the strong minimum principle and deduce 

that n = 0 or n > 0. The same is true for v. By Theorem 12.71 at least one of u, v 
vanishes. 

Assume for instance that u = 0. First, if p > 0 then —F{D‘^v) = 0, so by 
Lemma 13.51 we have v = C 2 > 0- Second, if p = 0 then F{D‘^v) = dv^^'^. Now, if 
d = 0 , then v = C 2 , while if d > 0, then u = 0 by Lemma ITT] because g + r > 1 . 

The same analysis can be done if u = 0. Hence, in all cases (u, v) is semitrivial. 
Finally, if r = 0, then it is clear that if u = 0, then v = 0, and vice versa. The last 
statements are obvious. □ 
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6 A priori estimates and an existence result in a bounded 
domain 


In this section we prove Theorem 12.111 We recall that we consider the following 
system with general lower order terms 


—T[u] = vl'v^\a{x)v‘^ — c{x)u^]^ +n, u), 
—F[v\ = v'^vP\b{x)u^ — d{x)v^]^ + h 2 {x^u^v), 
u = V = 0, 


X E n, 

X E n, 

X E 911, 


(55) 


where H is a bounded Lipschitz domain of M"", and satisfies (ff^) — {H 2 ). 


Proof of Theorem \2.11[ The proof of this theorem follows the same scheme as the 
proof of Theorems 6.1 and 6.2 of [28], where was assumed to be the Laplacian. We 
will not repeat the parts of the proof where it is similar to the one in the previous 
paper, and will only highlight the differences (most of which appear after the definition 
of the function S below). 

We consider the following parametrized version of system (|55p . 

—T[u]=F{t,x,u,v), X E n, 

< —P'[v]=G{t,x,u,v), X E n, (56) 

u = V = 0, X E dQ, 


where F and G are defined as in [28] . 

We perform exactly the same “blow-up” change of variables as in [28|. The only 
difference is that now the modified functions uj , Vj satisfy a system where appears 
the elliptic operator 

F{D'^Uj, XjDujjXj + Xju), 

and Xj —)• 0. By using the global ’“-estimates for Isaacs operators and Ascoli’s 
theorem, we can extract a subsequence of {uj,Vj) which converges in (the way to 
perform such a limit argument for fully nonlinear operators is described in extenso in 
[32]i. After the passage to the limit by Lemma [T8l we obtain a bounded nonnegative 
viscosity solution {U,V) of 

r -FiD^U, 0, xo) = [aoV^ - coW] , 

\ -F{D^V, 0, xo) = V^UP [boU^i - doV^] , 

either in R” or in a cone of R” which resembles locally the boundary of 12 at a 
neighborhood of some point of 912. In addition codo < in view of (f28]l . and 
U{0) > 2~“ > 0. By the strong maximum principle U > 0 everywhere. 

If the limiting domain is a cone, then Theorem 12.11 implies that U and V are 
proportional, hence both U and V are positive. This contradicts the assumption of 
Theorem 12.111 

So we can assume dSZD is set in R”. By performing a rotation in R”’ we may 
assume the operator in the left-hand side of (1571) is Fq^^o^ for any initially chosen 
orthogonal matrix Q. 

By the assumption of the theorem U and V cannot be both positive, so by The¬ 
orem 12.101 there is a constant C > 0 such that U = G and V = 0. Hence 


lim (iij, A) = (C, 0) locally uniformly on R”. (58) 

j^oo 
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Next, the exclusion of such semi-trivial rescaling limits is done exactly as in [28] if 
we observe that the definition of the principal eigenvalue of a fully nonlinear operator 
(recall Definition 13.3p trivially implies 






Thus the proof of the a priori bound (1321) is concluded. 

The proof of the existence part of Theorem 12.111 goes again like in [28| , replacing 
the Laplacian and its first eigenvalue by the fully nonlinear operator Fq^y and its hrst 
eigenvalue defined in Definition [331 also observing that, by [I] (see the discussion after 
Corollary 3.6 in that paper) that 


A+(-M+) < A+(-.T) = sup{A, —F + A satishes the maximum principle}. (59) 


We use the same fixed point theorem of Krasnoselskii and Benjamin as in [28] 
and the proof stays almost identical until the definition of the function S. Instead of 
setting S = y/uv as in [28], we now set 


S = min{n, uj. 


and we prove that, in the viscosity sense. 


- > (^-Ci)5 in D. (60) 

This implies that Xf{—F,Q) > A — Ci, which is in contradiction with the arbitrary 
choice of A (see (6.12), (6.13), (6.24) in [28] for more on the choice of A). 

We know that, in the viscosity sense, 

—F[u] > vJ'v'^\{a{x) + A)v‘^ — c(x)tt'^] + {A — Ci)u + A in D, 

—F[v\ > v'^vP\{b{x) + A)^^ — d{x)v^^^ + {A — Ci)v + A in D. 

Since we choose A such that A > max{||c||oo, ||d||oo}) we have Av'^ > c{x)u^ in 
the subset of D where v > u, and Au'^ > d{x)v'^ in the subset of D where u > v. 
Therefore 

—F[u] > {A — Ci)u = {A — Ci)S in D n {u > u}, 

—F[v] > {A — Ci)v = {A — Ci)S in D n {tt > ?;}. 

By Lemma 13.41 we get (I6UD . 

Finally, at the end of the proof, in order to show that the first hypothesis of 
the fixed point theorem (see Theorem 6.3 in [28] 1 is verified, we again argue by 
contradiction. Now, for any (small) (5 > 0 we can hnd a positive solution [u, v) 
with ||(u,u)|| < 5, of ([55]) with the right-hand side of this system multiplied by some 
rj G [0,1]. By using ([351) we obtain, with Ai = Aj^ (—7^,11) and for some cq > 0, 

—F[u] < -|- (Ai — eo) maxjti, ?;} < (^(maxlu, v}Y + (Ai — cq) maxjti, u}, 

—F[v\ < + (Ai — eo) maxjtt, uj < C{\n&yi{u, u})'^ -|- (Ai — cq) max{«, uj. 

As in Lemma 3.4 the maximum of subsolutions is a viscosity subsolution, so 

—T'[max{n, u}] < C(max{u, u})'^ -|- (Ai — eo)max{n, u} 

< ((7(5”^“^-|-Ai — eo) max{rt, uj in D 

Now, if we choose <5 sufficiently small, by ([5^ we get max{u, uj < 0, a contradiction. 
Theorem 12.Ill is proved. □ 
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